The present invention relates generally to interface devices for allowing humans to interface with computer systems, and more particularly to computer interface devices that allow the user to provide input to computer systems and provide force feedback to the user.
Computer systems are used extensively in many different industries to implement many applications, such as word processing, data management, simulations, games, and other tasks. A computer system typically displays a visual environment to a user on a display screen or other visual output device. Users can interact with the displayed environment to perform functions on the computer, play a game, experience a simulation or xe2x80x9cvirtual realityxe2x80x9d environment, use a computer aided design (CAD) system, or otherwise influence events or images depicted on the screen.
One visual environment that is particularly common is a graphical user interface (GUI). GUI""s present visual images which describe various graphical metaphors of a program or operating system implemented on the computer. Common GUI""s include the Windows(copyright) operating system from Microsoft Corporation and the System 7.5 operating system from Apple Computer, Inc. These interfaces allows a user to graphically select and manipulate functions of the operating system and application programs by using an input interface device. The user typically moves a user-controlled graphical object, such as a cursor or pointer, across a computer screen and onto other displayed graphical objects or predefined screen regions, and then inputs a command to execute a given selection or operation. The objects or regions (xe2x80x9ctargetsxe2x80x9d) can include, for example, icons, windows, pull-down menus, buttons, and scroll bars. Most GUI""s are currently 2-dimensional as displayed on a computer screen; however, three dimensional (3-D) GUI""s that present simulated 3-D environments on a 2-D screen can also be provided.
Other programs or environments that may provide user-controlled graphical objects such as a cursor include graphical xe2x80x9cweb pagesxe2x80x9d or other environments offered on the World Wide Web of the Internet, CAD programs, video games, virtual reality simulations, etc. In some graphical computer environments, the user may provide input to control a 3-D xe2x80x9cviewxe2x80x9d of the graphical environment, i.e., the user-controlled graphical xe2x80x9cobjectxe2x80x9d can be considered the view displayed on the video screen. The user can manipulate the interface device to move the view, as if moving a camera through which the user is looking. This type of graphical manipulation is common in CAD or 3-D virtual reality applications.
The user interaction with and manipulation of the computer environment is achieved using any of a variety of types of human-computer interface devices that are connected to the computer system controlling the displayed environment. In most systems, the computer updates the environment in response to the user""s manipulation of a user-manipulatable physical object (xe2x80x9cuser objectxe2x80x9d) that is included in the interface device, such as a mouse, joystick, etc. The computer provides feedback to the user utilizing the display screen and, typically, audio speakers.
Presently, there are two types of interface devices which use different sensing modes and different mappings to allow a user to interact with and manipulate a computer environment: isotonic sensing devices and isometric sensing devices. Isotonic sensing utilizes motion of a physical user object in physical space in predefined degrees of freedom to provide input to the computer. For example, a mouse is an isotonic controller often used to control a cursor in a GUI. The mouse may be moved in two degrees of freedom in the plane of a mousepad or other surface, and the cursor on the screen is moved directly in response to the movement of the mouse. A joystick is another example of an isotonic controller, where the movement of the stick in rotary or linear degrees of freedom of physical space is sensed and input to the computer. Other isotonic interface devices include trackballs, styluses and tablets, steering wheels, etc.
In contrast, isometric sensing utilizes a user""s force or pressure on the user object rather than the movement of the user object through physical space. The force magnitude and direction that the user exerts on the interface device is sensed and input to the computer to be used in the manipulation and interaction of the computer environment. For example, the xe2x80x9cSpace Ballxe2x80x9d from Space-Tec and the xe2x80x9cMagellanxe2x80x9d from Logitec are common isometric controllers. The Space Ball is a sphere having pressure sensors provided between the ball and the mounting surface. When the user touches the sphere, the sensor detects the direction and magnitude of force exerted by the touch. In ideal isometric sensing, there is no perceived deflection of the user object in response to the user""s pressure. However, if there is a small amount of deflection or movement in the user object perceived by the user, the sensing can be referred to as xe2x80x9celasticxe2x80x9d control. In many cases, isometric controllers are actually elastic controllers, since there is a small amount of deflection of the user object by which the magnitude of force is measured. Some users prefer this small deflection, as it provides some intuitive feedback as to the degree of pressure applied by the user. In many cases, elastic controllers have been found to induce smaller errors in user manipulation of computer objects than pure isometric controllers.
Human factors research has shown that isotonic controllers excel at position control tasks, while isometric controllers are more intuitive for use with rate control tasks. xe2x80x9cPosition controlxe2x80x9d refers to a direct mapping of the position of the user object with a user-controlled graphical object. For example, a cursor in a GUI is controlled with a mouse under a position control paradigm, since the cursor is moved a distance corresponding to the distance the mouse is moved. xe2x80x9cRate control,xe2x80x9d in contrast, refers to an indirect or abstract mapping of user object to graphical object. For example, scrolling text in a window or zooming to a larger view in a window of a GUI are rate control tasks, since the scrolling and zooming is not directly related to the position of a mouse. Similarly, the controlled velocity of a simulated vehicle is suitable for a rate control paradigm.
A problem with the current use of isotonic controllers, such as mice and trackballs, within GUI""s and other graphical environments is that both position control and rate control tasks are required in a single computer environment. For example, as described above, a GUI includes many position control tasks such as target acquisition, i.e., moving the cursor onto icons, buttons, menu items, text, etc. An isotonic controller such as a mouse is ideal for these types of interactions. However, other GUI interactions, such as scrolling text, zooming, panning/rotating a view, or sizing, are more appropriate for a rate control interface. To provide simple rate control interactions using an isotonic controller, several graphical metaphors have been invented. For example, in a position control interface, sliders are displayed which can be moved using a mouse to allow the scrolling of text, or a magnifying icon is selected to enable zooming. However, these graphical metaphors can often be awkward, especially in view of the ease of such rate control tasks when using an isometric or elastic controller. Indeed, some users who have a great need for rate control tasks such as scrolling and zooming may simultaneously use both an isotonic controller such as a mouse and an isometric controller such as a Space Ball to allow maximum ease of use in interacting with the computer environment. However, the use of two separate controllers for computer interactions is often awkward and inconveniencing for the user.
In addition, existing isometric controllers are limited in that they are only input devices and are not able to provide active force feedback to a user. The user is thus not able to experience force feedback when manipulating the isometric controller which can be provided when manipulating an isotonic controller such as a joystick. The user is therefore missing potentially valuable and interesting force information and assistance in executing tasks in a graphical environment when using a traditional isometric controller.
There are a few commercial examples of isotonic controllers that have additional control modes usable for rate control tasks. One example is the SoftMouse from Immersion Corporation that has been available for a number of years. This is a standard mouse controller that has an additional thumb wheel that can be rotated to control zoom functions. Another example is the forthcoming Intellimouse from Microsoft(copyright), which is a standard mouse controller having a finger wheel that may be rotated to control scrolling functions. Both of these are examples of poorly integrated multi-modal controllers because the additional modes are just add-ons to standard controllers. For example, add-on sensors are used to track the thumb wheels independently of standard mouse sensors. Also, different finger actions are required for each mode, e.g., moving a mouse to control one mode and turning a wheel to control another mode. And, like the isometric controllers, these types of controllers are input only controllers and are not able to provide computer-controlled output forces to a user.
What is needed is an integrated multi-modal controller where the same sensor and the same hand activities are used to implement multiple control modes. In addition, a seamless method to switch between modes is desirable to provide ease of use. Finally, a multi-modal device having force feedback provided by computer-controlled actuators in all available modes is needed for interactions of a user in a computer environment.
The present invention is directed to a force feedback interface which allows a user to provide both isotonic and isometric input to a host computer system. Isotonic input and force feedback is provided for position control tasks such as positioning a cursor or other graphical object, while isometric input is provided for easily performing rate control tasks.
More specifically, the present invention includes an interface device for providing isotonic and isometric input to a host computer system from a user. An interface device includes a user manipulatable physical object contacted by a user and movable in physical space. In the preferred embodiment, the physical object is a puck or mouse that can be moved in a planar workspace. A sensor detects the movement of the physical object in physical space and, preferably, an actuator applies output forces on the physical object. A mode selector is provided to select an isotonic control mode and an isometric control mode of the interface device. The isotonic mode provides input to the host computer system based on a position of the physical object in physical space with respect to a ground. The isometric mode provides input to the host computer system based on an input force applied by the user to the same physical object with respect to the same ground, where the input force is determined based on the movement detected by the sensor. In isometric mode, the input force applied by the user preferably opposes the output force applied by the actuator, and is preferably detected based on a measured deviation of the physical object in physical space from a locally-defined origin.
A method of the present invention similarly provides isotonic and isometric input from a user using a single interface device coupled to a host computer system that displays a graphical environment such as a graphical user interface (GUI). A selection of a control mode of the interface device is received, where the control mode is either isotonic control mode or isometric control mode. Isotonic input is provided to the host computer if the interface device is in isotonic mode, where the isotonic input is used by the host computer to update a position of a user-controlled graphical object in the graphical environment to correspond to a position of a user-manipulated physical object (such as a cursor) in provided degrees of freedom. The interface device is preferably in isotonic mode when the isometric mode is not active. Preferably, force sensations are applied to the physical object in isotonic mode based on interactions of the user-controlled graphical object in the graphical environment, where the force sensations assist and/or inform the user of interaction with graphical objects. A program function may be performed as indicated by the location of the cursor and a command gesture from the user.
Isometric input is provided to the host computer if the interface device is in isometric mode, where the isometric input is used by the host computer to control an isometric function of the graphical environment based on an input force applied by the user to the physical object. In a preferred embodiment, an indication is received to engage the isometric mode of the interface device. A local origin is defined with reference to a current position of the physical object in provided degrees of freedom. A deviation of the physical object from the local origin is determined, where this deviation is indicative of the user""s input force, and a resistive force is applied to the physical object opposing the deviation. The resistive force is preferably a restoring force having a magnitude proportional to a magnitude of the deviation from the local origin and a direction towards the local origin. The determined deviation is used to control an isometric function of an application program or operating system implemented by the host computer. The isometric function can include such tasks as scrolling a displayed document, panning a displayed view, or zooming a displayed view. Optionally, in isometric mode, the host computer may display movement of the user-controlled graphical object corresponding to the deviation of the physical object.
In one embodiment, the control mode may be selected by the user activating an input device such as a physical button provided on the physical object. Alternatively, the control mode can be selected based on an interaction between a user-controlled graphical object, such as a cursor, and a different graphical object displayed by the host computer in a graphical environment. This interaction can include moving the user-controlled graphical object against an xe2x80x9cisometric surfacexe2x80x9d of another graphical object. An indexing feature of the present invention allows the user to change the offset between the position of the physical object and the location of the cursor on the display screen by disabling the mapping in said isotonic mode between the user-controlled graphical object and the physical object. A safety switch may be included to deactivate output forces to the physical object when, e.g., the user removes the weight of his or her fingers from the physical object. In one embodiment, the safety switch and indexing feature are integrated into the same switch. In a described embodiment, a local microprocessor, separate from the host processor, is coupled to the interface device and may provide local control over sensing and outputting forces to relieve the computational burden on the host computer. Voice coil actuators or motors may be used as the actuators, and a linkage having a plurality of members can be included.
In some embodiments, an overlay force is added to the restoring force applied to the physical object in isometric mode. The overlay force can be a jolt force or vibration sensation to indicate to said user an event in the graphical environment, such as a page break of a scrolling document or a limit to a controlled range.
The method and apparatus of the present invention advantageously provides both isotonic and isometric sensing functionality in a single interface device. This allows the user to conveniently switch control modes to efficiently perform isotonic position control tasks and isometric rate control tasks in a graphical computer environment. Forces in the isotonic mode assist or inform the user in isotonic tasks, and the provision of overlay forces in isometric mode allows additional information to be presented to the user which was not possible in traditional isometric input devices. The safety switch and indexing features of the present invention allow a mouse-like force feedback interface to be implemented and manipulated with ease.
These and other advantages of the present invention will become apparent to those skilled in the art upon a reading of the following specification of the invention and a study of the several figures of the drawing.